Investigating CO2 Emissions and Renewable Energy Usage

Code

<<<<<<< HEAD

library(tidyverse)
library(broom)
library(knitr)
library(DT)
library(kableExtra)
library(gridExtra)
library(patchwork)

Error in library(patchwork): there is no package called 'patchwork'

Code

options(scipen = 99999)
energy <- readxl::read_xlsx(here::here("renewable_energy.xlsx"))
co2 <- readxl::read_xlsx(here::here("co2.xlsx"))

=======

library(plotly)
library(tidyverse)
library(broom)
library(knitr)
library(DT)
library(kableExtra)
library(gridExtra)
library(patchwork)
library(RColorBrewer)
options(scipen = 99999)
energy <- readxl::read_xlsx(here::here("renewable_energy.xlsx"))
co2 <- readxl::read_xlsx(here::here("co2.xlsx"))

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Code

<<<<<<< HEAD

convert_to_numeric <- function(x) {
  x <- str_replace_all(x, "k", "e3")
  x <- str_replace_all(x, "M", "e6")
  return(as.numeric(x))
}


co2_clean <- co2 |>
  select(country, `1989`:`2017`) |>
  mutate(across(.cols = `1989`:`2017`, 
                ~ convert_to_numeric(.))) |>
  pivot_longer(cols = !country,
               names_to = "year",
               values_to = "Total_CO2_Emissions")

energy_clean <- energy |>
  pivot_longer(cols = !country,
               names_to = "year",
               values_to = "Percent_Consumption_Renewable")

joined <- inner_join(co2_clean, energy_clean) |>
  drop_na()

joined |>
  distinct(year) |>
  count()

joined |>
  distinct(country) |>
  count()

master <- joined|>
  rename(Year = year) |>
  group_by(Year) |>
  summarize(`Average Emissions` = mean(`Total_CO2_Emissions`),
            `Average Percent Renewawable`
            = mean(`Percent_Consumption_Renewable`))

=======

convert_to_numeric <- function(x) {
  x <- str_replace_all(x, "k", "e3")
  x <- str_replace_all(x, "M", "e6")
  return(as.numeric(x))
}


co2_clean <- co2 |>
  select(country, `1989`:`2017`) |>
  mutate(across(.cols = `1989`:`2017`, 
                ~ convert_to_numeric(.))) |>
  pivot_longer(cols = !country,
               names_to = "year",
               values_to = "Total_CO2_Emissions")

energy_clean <- energy |>
  pivot_longer(cols = !country,
               names_to = "year",
               values_to = "Percent_Consumption_Renewable")

joined <- inner_join(co2_clean, energy_clean) |>
  drop_na()

joined |>
  distinct(year) |>
  count()

joined |>
  distinct(country) |>
  count()

master <- joined|>
  rename(Year = year) |>
  group_by(Year) |>
  mutate(Year = as.numeric(Year)) |>
  summarize(`Average Emissions` = mean(`Total_CO2_Emissions`),
            `Average Percent Renewable`
            = mean(`Percent_Consumption_Renewable`))

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

1 Introduction

Climate change is a global challenge jeopardizing humanity’s future. This purpose of this project is to investigate the relationship between CO2 emissions, one of the largest drivers of climate change creating greenhouse gasses, and the use of renewable energy such as solar, wind, or hydroelectric energy. The data to be used in this project incorporates data from two different sources, one measuring total carbon dioxide emissions per country over the years and one measuring proportion of energy use which is renewable over the years. Total_CO2_Emission is in 1000 tons of CO2 and Percent_Consumption_Renewables is a percentage of total energy use produced by renewable sources per country over the years. Because the data is collected on individual countries over the years, this investigation will primarily focus on the Average Emissions as well as the Average Percent Renewable of all the countries in a specific year. In the data of which the averages will be investigated, there are 207 distinct countries observed over 29 years. The relationship between Average Emissionsand Average Percent Renewable is hypothesized to be negative whereas humanity used more renewable energy on average there would be less average CO2 emissions. Through this investigation hopefully more information can be learned about climate change and potential solutions to this global problem affecting every organism on planet Earth.

2 Materials and Methods

The master data to be used for analysis in this project incorporates the averages for all countries over the years from two different data sets, one measuring the total carbon dioxide emissions, per country over the years and the other measuring the proportion of energy use classified as renewable per country over the years. The data sets were sourced from an online database titled Gapminder which has been a reliable provider of data since 2005, in hopes of promoting global sustainability through easily accessible information.

Before analysis, the data had to be properly cleaned and wrangled. The CO2 data set contained values in the form of 25.3k and 4.9M, signaling the units of thousands and millions. The first step in the data cleaning process was to replace these characters with numbers to create a numeric variable which calculations could be performed on. Once the data was of all the same type, the data was then pivoted to be in tidy form and the years of interest (1989 – 2017) were selected. Once pivoted, the data sets were joined by year and country to produce a clean master data set which shall be used for model fitting, plotting, and analysis. A final decision was made to drop all the Na’s in the master set due to a few reasons. The missing values typically occurred in the Average Percent Renewable variable across the earlier years measured and usually in countries with smaller populations. Whether these Na’s were included in the data due to lack of observation or for simply not having any renewable energy production or consumption it is unclear. Because the averages are being studied and the Na’s usually occured in smaller countries, there was still a large enough sample size to average over once those observations were dropped. Other forms of imputation were contemplated such as cell-mean imputation but not conducted due to fears of introducing bias to the data.

Linear regression, a method involving predicting the values of one variable, based on another, through producing a straight line minimizing the value for the sum of squared residuals, was used to create and predict a model. All the data cleaning, models, and subsequent analysis was conducted using R code.

Code

datatable((head(master, n = 50)),
          caption = 'Interactive Preview of Data Set')

<<<<<<< HEAD <<<<<<< HEAD

=======

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

3 Analysis and Discussion of Model

Code

<<<<<<< HEAD

energy_emissions_model <- lm(`Average Emissions` ~ 
                             `Average Percent Renewawable`,
                             data = master)
summary(energy_emissions_model)
tidy(energy_emissions_model)

=======

energy_emissions_model <- lm(`Average Emissions` ~ 
                             `Average Percent Renewable`,
                             data = master)
summary(energy_emissions_model)
tidy(energy_emissions_model)

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Regression Equation:

\[ \hat{y} = 693309.6 - 17770.2 * Average\,Percent\,Renewable \]

Above is the simple linear regression equation based on the model predicting the response variable, Average Emissions (\(\hat{y}\)), by the explanatory variable, Average Percent Renewable. The coefficient on Average Percent Renewable is extremely negative sitting at -17770.2 meaning that for each one percent increase in the Average Percent Renewable energy used the average amount of CO2 emissions in thousands of tons decreases by 17770.2. The slope coefficient of 693309.6 means that when the Average Percent Renewable is zero such that there is no renewable energy being used at all on average, the predicted Average Emissions would be 693309.6 thousand tons.

Code

<<<<<<< HEAD

# Plot 1
raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewawable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 
  geom_smooth(method = "lm") +
  theme(legend.position = "none") +
  labs(x = " Average Renewable Energy Consumption (%)", 
       y = "", 
       title = "Relationship between Renewable Energy Usage and CO2 Emissions", 
       subtitle = "Average CO2 Emissions (1000 tonnes)") +
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'),
        plot.subtitle = element_text(size = 10),
        axis.title.x = element_text(size = 10))

raw_graph

=======

# Plot 1
raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 
  geom_smooth(method = "lm") +
  theme(legend.position = "none") +
  labs(x = " Average Renewable Energy Consumption (%)", 
       y = "", 
       title = "Relationship between Renewable Energy Usage and CO2 Emissions", 
       subtitle = "Average CO2 Emissions (1000 tonnes)") +
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'),
        plot.subtitle = element_text(size = 10),
        axis.title.x = element_text(size = 10))

raw_graph

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

The graph above demonstrates the relationship between Average Emissions and Average Percent Renewable. The distribution illustrates a negative linear relationship, where the points are relatively close to the plotted regression line with little deviation and noise. There are little to no unusual observations. This illustration is consistent with the hypothesis that as the Average Percent Renewable increases the Average Emissions decreases at a significant rate.

Code

<<<<<<< HEAD

co2_by_year_graph <- master |>
  ggplot(aes(x = Year , y = `Average Emissions`)) +
  geom_point() +
  scale_x_discrete(guide = guide_axis(n.dodge=2)) +
  labs(x = "Year", 
       y = "", 
       title = "Average CO2 Emissions Over Time", 
       subtitle = "Average CO2 Emissions (1000 tonnes)") +
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

energy_by_year_graph <- master |>
  ggplot(aes(x = Year, y = `Average Percent Renewawable`)) +
  geom_point() +
   scale_x_discrete(guide = guide_axis(n.dodge=2)) +
    labs(x = "Year", 
       y = "", 
       title = "Average Percentage of Renewable Energy Over Time",
       subtitle = "Average Renewable Energy Consumption (%)") +
      theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

grid.arrange(co2_by_year_graph, energy_by_year_graph)

=======

co2_by_year_graph <- plot_ly(
  master, 
  x = ~ Year, 
  y = ~ `Average Emissions`,
  type = 'scatter',
  marker = list(
    color = 'red'))

co2_by_year_graph <- co2_by_year_graph |>
  layout(title = 'Average Emissions Over Time',
         yaxis = list(
           title = 'Average Emissions (1000 tons)',
           titlefont = list(
             size = 14),
           xaxis = list(title = 'Year',
                        titlefont = list(size = 14))))

energy_by_year_graph <- plot_ly(
  master, 
  x = ~ Year, 
  y = ~ `Average Percent Renewable`,
  marker = list(
    color = 'green'),
  type = 'scatter')

energy_by_year_graph <- energy_by_year_graph |>
  layout(title = 'Average Percent Renewable Over Time',
         yaxis = list(
           title = 'Average Percent Renewable',
           titlefont = list(
             size = 14)),
         xaxis = list(
           title = 'Year',
           titlefont = list(size = 14)))


co2_by_year_graph

<<<<<<< HEAD

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

Code

energy_by_year_graph

As shown by the two distributions of Average Emissions and Average Percent Renewable over time, the relationship follows a negative relationship, but perhaps not as expected. Average Percent Renewable is decreasing over the years while Average Emissions is increasing which still illustrates a negative relationship. As time goes on it makes sense as to why Average Emissions is increasing, because of extreme population growth and growing demand for production but Average Percent Renewable has shockingly been declining in recent years. While this likely has something to due to the varying definition of renewable energy, for example whether or not nuclear energy is truly renewable, it is surprising that as technology develops renewable energy use does not. This signifies that humanity needs to increase the renewable energy production and usage on average in order to reduce the carbon footprint and preserve nature for future generations.

Model Fit:

Code

<<<<<<< HEAD

energy_emissions_model |>
  augment() |>
  summarize(`Variance of Fitted` = var(.fitted),
            `Variance of Residuals` = var(.resid),
            `Variance of Average CO2 Emissions` = var(`Average Emissions`)) |>
  kable(caption = 'Model Fit',
        digits = 3,
        format.args = list(big.mark = ",")) |>
  kable_styling(bootstrap_options = c('striped', 'bordered'))

=======

energy_emissions_model |>
  augment() |>
  summarize(`Variance of Fitted` = var(.fitted),
            `Variance of Residuals` = var(.resid),
            `Variance of Average CO2 Emissions` = var(`Average Emissions`)) |>
  kable(caption = 'Model Fit',
        digits = 3,
        format.args = list(big.mark = ",")) |>
  kable_styling(bootstrap_options = c('striped', 'bordered'))

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Model Fit

Variance of Fitted	Variance of Residuals	Variance of Average CO2 Emissions
415,649,404	46,532,518	462,181,923

The proportion of variability in the response values that was accounted for by the model, \(R^{2}\), was very large at about at about 89.93 percent. This suggests a good quality model, where a lot, about 89%, of the variation in the response, Average Emissions is explained by the explanatory variable, Average Percent Renewable. This suggests that a high proportion of variability in response is accounted for by the linear model and there are not many other large factors influencing emissions.

Code

<<<<<<< HEAD

energy_emissions_model |> 
  augment() |> 
  ggplot(aes(x=.fitted, y = .resid)) +
  geom_point() +
  labs(y = '',
       subtitle = 'Residuals',
       x = 'Fitted Values',
       title = 'Relationship between Residual and Fitted Values') +
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

=======

energy_emissions_model |> 
  augment() |> 
  ggplot(aes(x=.fitted, y = .resid)) +
  geom_point() +
  labs(y = '',
       subtitle = 'Residuals',
       x = 'Fitted Values',
       title = 'Relationship between Residual and Fitted Values') +
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Simulation:

Simulation is a critical technology to develop planning and explore models to optimized decisions making (de Paula Ferreira et al., 2020).

In this part, we will perform a basic linear model simulation to see how well the model is with the presetting conditions, such as adding normally distributed error to the linear regression line.

The basic procedure in this study is:

1. Make a linear regression fit model for the observed data (already done in the previous part).
2. Assume the model is right. Add generated error to the linear regression model (we generated normally distributed error for this study).
3. Getting the simulated data, and compare with the observed data (by generating value distribution graphs, scatterplots of the relationships modeled and observed, and y=x plot).
4. Check and interpret the simulated \(R^2\) value.
5. Iterating and generating simulated data sets.
6. Check, interpret, and plot the simulated \(R^2\) values for the simulated data sets.

Simulation for a single data set:

Code

<<<<<<< HEAD

noise <- function(x, mean = 0, sd){
  x + rnorm(length(x), 
            mean, 
            sd)
}

=======

noise <- function(x, mean = 0, sd){
  x + rnorm(length(x), 
            mean, 
            sd)
}

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Code

<<<<<<< HEAD

master_predict <- predict(energy_emissions_model)
master_sigma <- sigma(energy_emissions_model)

sim_response <- tibble(sim_emissions = noise(master_predict, 
                                          sd = master_sigma))






raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewawable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 
  geom_smooth(method = "lm") +
  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different contries", 
       subtitle = "Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

=======

master_predict <- predict(energy_emissions_model)
master_sigma <- sigma(energy_emissions_model)

sim_response <- tibble(sim_emissions = noise(master_predict, 
                                          sd = master_sigma))

raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 
  geom_smooth(method = "lm") +
  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different contries", 
       subtitle = "Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Code

<<<<<<< HEAD

obs_emissions <- master |>
  ggplot(aes(x = `Average Emissions`)) +
  geom_histogram(binwidth = 3000) +
  labs(x = "Observed Emissions",
       y = "",
       subtitle = "Count") +
  theme_bw()


sim_emissions_graph <- sim_response |>
  ggplot(aes(x = sim_emissions)) +
  geom_histogram(binwidth = 3500) +
  labs(x = "Simulated Emissions",
       y = "",
       subtitle = "Count") +
  theme_bw()

obs_emissions + sim_emissions_graph

Error in `ggplot_add()`:
! Can't add `sim_emissions_graph` to a <ggplot> object.

=======

obs_emissions <- master |>
  ggplot(aes(x = `Average Emissions`)) +
  geom_histogram(binwidth = 3000) +
  labs(x = "Observed Emissions",
       y = "",
       subtitle = "Count",
       title = "Worldwidely Yearly Emission Counts") +
  theme_bw()+
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))


sim_emissions_graph <- sim_response |>
  ggplot(aes(x = sim_emissions)) +
  geom_histogram(binwidth = 3500) +
  labs(x = "Simulated Emissions",
       y = "",
       subtitle = "Count",
       title = "Simulated Worldwidely Yearly Emission Counts") +
  theme_bw()+
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

obs_emissions + sim_emissions_graph

<<<<<<< HEAD

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

We can see the differences between observed and simulated data from the distribution visualization. When we first look at the two graphs, they seem not quite similar. However, when we take a close look, the concentrated value ranges are similar.

The left graph described the distribution of the observed yearly emissions data counts world widely. We can see that the most data are within \(1.1\times10^6\) tonnes to \(1.2\times10^6\) tonnes and \(1.6\times10^6\) tonnes to \(1.7\times10^6\) tonnes. The simulated yearly emissions data are also concentrated in a range of \(1.1\times10^6\) tonnes to \(1.2\times10^6\) tonnes and \(1.5\times10^6\) tonnes to \(1.7\times10^6\) tonnes.

Code

<<<<<<< HEAD

sim_data <- master |> 
  filter(!is.na(`Average Emissions`), 
         !is.na(`Average Percent Renewawable`)
         ) |> 
  select(`Average Emissions`, `Average Percent Renewawable`) |> 
  bind_cols(sim_response)


raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewawable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 

  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Observed Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different contries", 
       subtitle = "Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

sim_master_graph <- sim_data |> 
ggplot(aes(x = `Average Percent Renewawable`, 
           y = sim_emissions)
       ) +
  geom_point() + 
  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Simulated Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different contries", 
       subtitle = "Simulated Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

raw_graph + sim_master_graph

Error in `ggplot_add()`:
! Can't add `sim_master_graph` to a <ggplot> object.

=======

sim_data <- master |> 
  filter(!is.na(`Average Emissions`), 
         !is.na(`Average Percent Renewable`)
         ) |> 
  select(`Average Emissions`, `Average Percent Renewable`) |> 
  bind_cols(sim_response)


raw_graph <- master |> 
ggplot(aes(x = `Average Percent Renewable`, 
           y = `Average Emissions`)
       ) +
  geom_point() + 

  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Observed Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different contries", 
       subtitle = "Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

sim_master_graph <- sim_data |> 
ggplot(aes(x = `Average Percent Renewable`, 
           y = sim_emissions)
       ) +
  geom_point() + 
  theme(legend.position = "none") +
  labs(x = " Avg Renewable energy consumption percentage", 
       y = "", 
       title = "Simulated Relationships between Avg Renewable Energy Consumption and \nAvg CO2 Emissions among different countries", 
       subtitle = "Simulated Avg CO2 Emissions/1000 tonnes") +
  theme(plot.title.position = "plot")

raw_graph + sim_master_graph

<<<<<<< HEAD

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

In these two graphs, we plotted the relationships between simulated/ observed average renewable energy consumption and average CO2 emissions among different countries. We can see the the simulated data in the right graph is more close to a straight line, which is the modeled linear regression line.

Code

<<<<<<< HEAD

# Check the similarity between the simulated data and observed data
sim_data |> 
  ggplot(aes(x = sim_emissions, 
             y = `Average Emissions`)
         ) + 
  geom_point() + 
   labs(x = "Simulated Avg CO2 Emissions/1000 tonnes", 
        y = "",
        subtitle = "Avg CO2 Emissions/1000 tonnes" ) + 
  geom_abline(slope = 1,
              intercept = 0, 
              color = "steelblue",
              linetype = "dashed",
              lwd = 1.5) +
  theme_bw()

=======

# Check the similarity between the simulated data and observed data
sim_data |> 
  ggplot(aes(x = sim_emissions, 
             y = `Average Emissions`)
         ) + 
  geom_point() + 
   labs(x = "Simulated Avg CO2 Emissions/1000 tonnes", 
        y = "",
        title = "The Similarity between Observed Data and Simulated Data",
        subtitle = "Avg CO2 Emissions/1000 tonnes" ) + 
  geom_abline(slope = 1,
              intercept = 0, 
              color = "steelblue",
              linetype = "dashed",
              lwd = 1.5) +
  theme_bw()+
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

<<<<<<< HEAD

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

In this graph, we can check the similarity between the data from observed data set and the simulated data set. Ideally, the data should fall on the blue dashed line, which is y=x line. The dots/ data above the blue line over estimated/ simulated. The dots/ data below the blue line is under estimated/ simulated. In this study, the observed data is more under estimated. Generally speaking, the dots are close to the y=x line. In this case, there is a strong relationship between the observed values and simulated values.

p value

Code

<<<<<<< HEAD

# check how the simulated data fit the observed data, especially the R square.
lm(`Average Emissions` ~ sim_emissions, 
   data = sim_data
   ) |> 
  glance() 

# A tibble: 1 × 12
  r.squared adj.r.squa…¹ sigma stati…²  p.value    df logLik   AIC   BIC devia…³
      <dbl>        <dbl> <dbl>   <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>   <dbl>
1     0.821        0.814 9261.    124. 1.36e-11     1  -305.  616.  620.  2.32e9
# … with 2 more variables: df.residual <int>, nobs <int>, and abbreviated
#   variable names ¹​adj.r.squared, ²​statistic, ³​deviance

=======

# check how the simulated data fit the observed data, especially the P value and R square.
p_value<-lm(`Average Emissions` ~ sim_emissions, 
   data = sim_data
   ) |> 
  glance() |>
  select(p.value)|>
  pull()
p_value

<<<<<<< HEAD

                value 
0.0000000000002696793 

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

               value 
0.000000000001536224 

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

r square

Code

<<<<<<< HEAD

# get the R squared for the simulated data fit for observed data.
sim_r2 <- lm(`Average Emissions` ~ sim_emissions, 
             data = sim_data
             ) |> 
  glance() |> 
  select(r.squared) |> 
  pull()
sim_r2

[1] 0.8210543

=======

# get the R squared for the simulated data fit for observed data.
sim_r2 <- lm(`Average Emissions` ~ sim_emissions, 
             data = sim_data
             ) |> 
  glance() |> 
  select(r.squared) |> 
  pull()
sim_r2

<<<<<<< HEAD

[1] 0.8659149

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

[1] 0.8475827

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

p value is extremely small (p<0.05), which indicates the data is statistically significant.

\(R^2\) is 0.803297, which means the simulated data can explain around 80% variation of the observed data. In this case, we suppose the one time simulation till now did a fairly good job.

Simulation for a 1000 data sets:

Code

<<<<<<< HEAD

# Created 1000 simulated dataset
nsims <- 1000
sims <- map_dfc(.x = 1:nsims,
                .f = ~ tibble(sim = noise(master_predict, 
                                          sd = master_sigma)
                              )
                )

head(sims)

# A tibble: 6 × 1,000
  sim...1 sim...2 sim...3 sim...4 sim...5 sim...6 sim...7 sim...8 sim...9
    <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>
1 124107. 111736. 112793. 119910. 103948. 110044. 113407. 102446. 119776.
2 123417. 106374. 108435. 103987. 101951. 110609. 119170. 115403. 108647.
3 117173. 120500. 115167. 114152. 111252. 106961. 109460. 107555. 111107.
4 110998. 116846. 113926. 111626. 112144. 115220. 115971. 109659. 101394.
5 111051. 100945.  98565.  98743. 108427. 102417. 117136. 105620.  98257.
6 107355.  89757. 108215. 105845.  87102. 105102.  93000. 100241.  98826.
# … with 991 more variables: sim...10 <dbl>, sim...11 <dbl>, sim...12 <dbl>,
#   sim...13 <dbl>, sim...14 <dbl>, sim...15 <dbl>, sim...16 <dbl>,
#   sim...17 <dbl>, sim...18 <dbl>, sim...19 <dbl>, sim...20 <dbl>,
#   sim...21 <dbl>, sim...22 <dbl>, sim...23 <dbl>, sim...24 <dbl>,
#   sim...25 <dbl>, sim...26 <dbl>, sim...27 <dbl>, sim...28 <dbl>,
#   sim...29 <dbl>, sim...30 <dbl>, sim...31 <dbl>, sim...32 <dbl>,
#   sim...33 <dbl>, sim...34 <dbl>, sim...35 <dbl>, sim...36 <dbl>, …

=======

# Created 1000 simulated dataset
nsims <- 1000
sims <- map_dfc(.x = 1:nsims,
                .f = ~ tibble(sim = noise(master_predict, 
                                          sd = master_sigma)
                              )
                )

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Since the normally distributed error is randomly added to the linear regression model line. If we iterate for more times, the result will be more stable. In this way, we can get the entire group of interest, rather than a single sample of that group.

In the following steps, we got 1000 simulated data sets and combined the observed data set to a new full data sets. Then, we also got the \(R^2\) of 1000 simulated data sets and plotted the distribution of them.

Code

<<<<<<< HEAD

# clean the colnames
colnames(sims) <- colnames(sims) |> 
  str_replace(pattern = "\\.\\.\\.",
                  replace = "_")
head(sims)

# A tibble: 6 × 1,000
    sim_1   sim_2  sim_3  sim_4  sim_5  sim_6  sim_7  sim_8  sim_9 sim_10 sim_11
    <dbl>   <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>
1 124107. 111736. 1.13e5 1.20e5 1.04e5 1.10e5 1.13e5 1.02e5 1.20e5 1.18e5 1.23e5
2 123417. 106374. 1.08e5 1.04e5 1.02e5 1.11e5 1.19e5 1.15e5 1.09e5 1.12e5 1.14e5
3 117173. 120500. 1.15e5 1.14e5 1.11e5 1.07e5 1.09e5 1.08e5 1.11e5 1.16e5 1.14e5
4 110998. 116846. 1.14e5 1.12e5 1.12e5 1.15e5 1.16e5 1.10e5 1.01e5 1.13e5 1.27e5
5 111051. 100945. 9.86e4 9.87e4 1.08e5 1.02e5 1.17e5 1.06e5 9.83e4 1.13e5 1.02e5
6 107355.  89757. 1.08e5 1.06e5 8.71e4 1.05e5 9.30e4 1.00e5 9.88e4 1.08e5 1.11e5
# … with 989 more variables: sim_12 <dbl>, sim_13 <dbl>, sim_14 <dbl>,
#   sim_15 <dbl>, sim_16 <dbl>, sim_17 <dbl>, sim_18 <dbl>, sim_19 <dbl>,
#   sim_20 <dbl>, sim_21 <dbl>, sim_22 <dbl>, sim_23 <dbl>, sim_24 <dbl>,
#   sim_25 <dbl>, sim_26 <dbl>, sim_27 <dbl>, sim_28 <dbl>, sim_29 <dbl>,
#   sim_30 <dbl>, sim_31 <dbl>, sim_32 <dbl>, sim_33 <dbl>, sim_34 <dbl>,
#   sim_35 <dbl>, sim_36 <dbl>, sim_37 <dbl>, sim_38 <dbl>, sim_39 <dbl>,
#   sim_40 <dbl>, sim_41 <dbl>, sim_42 <dbl>, sim_43 <dbl>, sim_44 <dbl>, …

=======

# clean the colnames
colnames(sims) <- colnames(sims) |> 
  str_replace(pattern = "\\.\\.\\.",
                  replace = "_")

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

Original Average Emission data with 1000 simulated datasets

Code

<<<<<<< HEAD

# bind 1000 simulated dataset and the observed dataset together
sims <- master |> 
  filter(!is.na(`Average Emissions`), 
         !is.na(`Average Percent Renewawable`)) |> 
  select(`Average Emissions`) |> 
  bind_cols(sims)

head(sims)

# A tibble: 6 × 1,001
  Averag…¹  sim_1  sim_2  sim_3  sim_4  sim_5  sim_6  sim_7  sim_8  sim_9 sim_10
     <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>
1  114253. 1.24e5 1.12e5 1.13e5 1.20e5 1.04e5 1.10e5 1.13e5 1.02e5 1.20e5 1.18e5
2  113712. 1.23e5 1.06e5 1.08e5 1.04e5 1.02e5 1.11e5 1.19e5 1.15e5 1.09e5 1.12e5
3  110629. 1.17e5 1.20e5 1.15e5 1.14e5 1.11e5 1.07e5 1.09e5 1.08e5 1.11e5 1.16e5
4  111783. 1.11e5 1.17e5 1.14e5 1.12e5 1.12e5 1.15e5 1.16e5 1.10e5 1.01e5 1.13e5
5  111874. 1.11e5 1.01e5 9.86e4 9.87e4 1.08e5 1.02e5 1.17e5 1.06e5 9.83e4 1.13e5
6  113577. 1.07e5 8.98e4 1.08e5 1.06e5 8.71e4 1.05e5 9.30e4 1.00e5 9.88e4 1.08e5
# … with 990 more variables: sim_11 <dbl>, sim_12 <dbl>, sim_13 <dbl>,
#   sim_14 <dbl>, sim_15 <dbl>, sim_16 <dbl>, sim_17 <dbl>, sim_18 <dbl>,
#   sim_19 <dbl>, sim_20 <dbl>, sim_21 <dbl>, sim_22 <dbl>, sim_23 <dbl>,
#   sim_24 <dbl>, sim_25 <dbl>, sim_26 <dbl>, sim_27 <dbl>, sim_28 <dbl>,
#   sim_29 <dbl>, sim_30 <dbl>, sim_31 <dbl>, sim_32 <dbl>, sim_33 <dbl>,
#   sim_34 <dbl>, sim_35 <dbl>, sim_36 <dbl>, sim_37 <dbl>, sim_38 <dbl>,
#   sim_39 <dbl>, sim_40 <dbl>, sim_41 <dbl>, sim_42 <dbl>, sim_43 <dbl>, …

=======

# bind 1000 simulated dataset and the observed dataset together
sims <- master |> 
  filter(!is.na(`Average Emissions`), 
         !is.na(`Average Percent Renewable`)) |> 
  select(`Average Emissions`) |> 
  bind_cols(sims)

head(sims)

<<<<<<< HEAD

# A tibble: 6 x 1,001
  `Average Emissions`   sim_1   sim_2   sim_3  sim_4  sim_5  sim_6  sim_7  sim_8
                <dbl>   <dbl>   <dbl>   <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>
1             114253. 121246. 107511. 105641. 1.03e5 1.05e5 9.24e4 1.05e5 1.16e5
2             113712. 112424. 122432. 119376. 1.07e5 1.19e5 1.12e5 1.19e5 1.04e5
3             110629. 113543. 116969. 116728. 1.13e5 1.13e5 1.06e5 1.14e5 1.23e5
4             111783. 124789. 104035. 109726. 1.11e5 9.87e4 1.11e5 1.08e5 1.16e5
5             111874. 113130.  97470.  96999. 1.03e5 1.05e5 8.73e4 1.16e5 9.94e4
6             113577. 104132.  99628.  91402. 1.01e5 9.04e4 9.81e4 1.02e5 1.00e5
# ... with 992 more variables: sim_9 <dbl>, sim_10 <dbl>, sim_11 <dbl>,
#   sim_12 <dbl>, sim_13 <dbl>, sim_14 <dbl>, sim_15 <dbl>, sim_16 <dbl>,
#   sim_17 <dbl>, sim_18 <dbl>, sim_19 <dbl>, sim_20 <dbl>, sim_21 <dbl>,
#   sim_22 <dbl>, sim_23 <dbl>, sim_24 <dbl>, sim_25 <dbl>, sim_26 <dbl>,
#   sim_27 <dbl>, sim_28 <dbl>, sim_29 <dbl>, sim_30 <dbl>, sim_31 <dbl>,
#   sim_32 <dbl>, sim_33 <dbl>, sim_34 <dbl>, sim_35 <dbl>, sim_36 <dbl>,
#   sim_37 <dbl>, sim_38 <dbl>, sim_39 <dbl>, sim_40 <dbl>, sim_41 <dbl>, ...

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd =======

# A tibble: 6 × 1,001
  Averag…¹  sim_1  sim_2  sim_3  sim_4  sim_5  sim_6  sim_7  sim_8  sim_9 sim_10
     <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>  <dbl>
1  114253. 1.14e5 1.17e5 1.06e5 1.10e5 1.19e5 1.10e5 1.03e5 1.07e5 1.15e5 1.15e5
2  113712. 1.17e5 1.07e5 1.18e5 1.09e5 1.20e5 9.74e4 1.08e5 1.07e5 1.10e5 9.94e4
3  110629. 1.04e5 1.13e5 1.07e5 1.13e5 1.15e5 1.14e5 1.10e5 1.17e5 1.10e5 1.06e5
4  111783. 1.08e5 1.01e5 1.07e5 9.31e4 1.16e5 1.14e5 1.13e5 1.12e5 1.08e5 1.14e5
5  111874. 1.13e5 1.03e5 1.08e5 1.08e5 9.87e4 1.07e5 1.14e5 1.07e5 1.14e5 1.02e5
6  113577. 1.12e5 1.14e5 1.14e5 1.05e5 1.19e5 1.08e5 1.07e5 9.84e4 1.12e5 1.02e5
# … with 990 more variables: sim_11 <dbl>, sim_12 <dbl>, sim_13 <dbl>,
#   sim_14 <dbl>, sim_15 <dbl>, sim_16 <dbl>, sim_17 <dbl>, sim_18 <dbl>,
#   sim_19 <dbl>, sim_20 <dbl>, sim_21 <dbl>, sim_22 <dbl>, sim_23 <dbl>,
#   sim_24 <dbl>, sim_25 <dbl>, sim_26 <dbl>, sim_27 <dbl>, sim_28 <dbl>,
#   sim_29 <dbl>, sim_30 <dbl>, sim_31 <dbl>, sim_32 <dbl>, sim_33 <dbl>,
#   sim_34 <dbl>, sim_35 <dbl>, sim_36 <dbl>, sim_37 <dbl>, sim_38 <dbl>,
#   sim_39 <dbl>, sim_40 <dbl>, sim_41 <dbl>, sim_42 <dbl>, sim_43 <dbl>, …

>>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

R squared for 1000 simulated datasets.

Code

<<<<<<< HEAD

# mapping to get 1000 simulated R squared.
sim_r_sq <- sims |> 
  map(~ lm(`Average Emissions` ~ .x, data = sims)) |> 
  map(glance) |> 
  map_dbl(~ .x$r.squared)

head(sim_r_sq)

Average Emissions             sim_1             sim_2             sim_3 
        1.0000000         0.8562973         0.7550128         0.8842073 
            sim_4             sim_5 
        0.8681675         0.7613779 

=======

# mapping to get 1000 simulated R squared.
sim_r_sq <- sims |> 
  map(~ lm(`Average Emissions` ~ .x, data = sims)) |> 
  map(glance) |> 
  map_dbl(~ .x$r.squared)

head(sim_r_sq)

Average Emissions             sim_1             sim_2             sim_3 
        1.0000000         0.8030574         0.8284892         0.8438176 
            sim_4             sim_5 
<<<<<<< HEAD
        0.8151787         0.7634048 

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd ======= 0.7560763 0.8570422 >>>>>>> 2d6ac341fee300eb63ebafc8e1983aa7aacb6262

Simulated R squared distribution

Code

<<<<<<< HEAD

# to see the distribution of the 1000 simulated R square.
tibble(sims = sim_r_sq) |> 
  ggplot(aes(x = sims)) + 
  geom_histogram(binwidth = 0.025) +
  labs(x = expression("Simulated"~ R^2),
       y = "",
       subtitle = "Number of Simulated Models")

Code

#The distribution of these values will tell if our assumed model does a good job of producing data similar to what was observed. If the model produces data similar to what was observed, we would expect values near 1.
# Our model is R square is around 0.8.

=======

# to see the distribution of the 1000 simulated R square.
tibble(sims = sim_r_sq) |> 
  ggplot(aes(x = sims)) + 
  geom_histogram(binwidth = 0.025) +
  labs(x = expression("Simulated"~ R^2),
       y = "",
       subtitle = "Number of Simulated Models",
       title = "1000 Simulated R-Squared Distribution")+
  theme(plot.title = element_text(hjust = 0.5, face = 'bold'))

Code

#The distribution of these values will tell if our assumed model does a good job of producing data similar to what was observed. If the model produces data similar to what was observed, we would expect values near 1.
# Our model is R square is around 0.8.

>>>>>>> 5b7bbb9045f9cf8201a690ba8e5c5c84ba477ebd

We can see from the distribution of the 1000 simulated \(R^2\) graph that the \(R^2\) have the highest frequency around 0.8, which is in accordance with the single simulated data \(R^2\).

4 References

1. https://www.gapminder.org/data/ https://plotly.com/r/line-and-scatter/#custom-color-scales

​2. de Paula Ferreira, W., Armellini, F., & De Santa-Eulalia, L. A. (2020). Simulation in industry 4.0: A state-of-the-art review. Computers & Industrial Engineering, 149, 106868. https://doi.org/10.1016/J.CIE.2020.106868